Approximate Message Passing

In this note, I summarize Sections 5.1 and 5.2 of Arian Maleki’s PhD thesis. 1 Notation We denote scalars by small letters e.g. a, b, c, . . ., vectors by boldface small letters e.g. λ,α,x, . . ., matrices by boldface capital letter e.g. A,B,C, . . ., (subsets of) natural numbers by capital letters e.g. N,M, . . .. We denote i’th element of a vector a by ai and (i, j)’th entry of a matrix A by Aij . We denote the i’th column (or row) of A by A:,i (or Ai,:). We use Aa,−i (or A−a, i to refer to the a’th row (or i’th column) without the element Aa,i. Also, A T denote the transpose of a matrix A. 2 Basis Pursuit Problem Given measurements y of length n and matrix A of size n×N , we wish to compute s which is the minimizer of Eq. 1. This is known as the basis pursuit problem. Here, || · ||1 is the l1-norm. A version of this problem where we allow for errors in the measurements is called basis pursuit denoising problem (aka LASSO), shown in Eq. 2. Here, || · ||2 is the l2-norm. BP: min s ||s||1, s.t. y = As (1) BPDN: min s λ||s||1 + 12 ||y −As|| 2 2 (2) 3 Posterior Distribution Consider the following posterior distributions in Eq. 3, where the prior distribution p(si) is the Laplace distribution and the likelihood p(ya|s,Aa,:) is the Dirac distribution. p(s|y) ∝ N ∏